On the Performance of Kernel Classes

where (Ω,μ) is a probability space. The kernel K is used to generate a Hilbert space, known as a reproducing kernel Hilbert space, whose unit ball is the class of functions we investigate. Recall that if K is a positive definite function K : Ω×Ω → R, then by Mercer’s Theorem there is an orthonormal basis (φi)i=1 of L2(μ) such that μ× μ almost surely, K(x,y) = ∑i=1 λiφi(x)φi(y), where (λi)i=1 is the sequence of eigenvalues of TK (arranged in a non-increasing order) and φi is the eigenvector corresponding to λi. Let HK be the set of functions of the form ∑i=1 aiK(xi, ·), where xi ∈ Ω and ai ∈ R satisfy that ∑i, j=1 aia jK(xi,x j) ≤ 1. One can show that this so-called kernel class HK is the unit ball in the reproducing kernel Hilbert space defined by the integral operator, and that for every f ∈ HK, ‖ f‖∞ ≤ ‖K‖∞. An alternative way to define the reproducing kernel Hilbert space is via the feature map. Indeed, if we define Φ : Ω → l2 by Φ(x) = (√ λiφi(x) ∞ i=1, then